guest-agent: fix v1 leftovers and pin the two untested v1 methods - #1118
Merged
Conversation
A post-merge audit of the v1 surface turned up five more comments in the species #1116 was already cleaning up, three documentation gaps, and two of the six v1 methods with no handler-level test. Comments naming things that do not exist: `GetAttestationForAppKey`'s doc still credited "the v1 method" with sharing its report data, and its vector test still said `WorkerV1` wraps those bytes in an attestation -- v1 ships neither. Four more spots wrote `WorkerV1` as a service name; the service is `Worker` in package `dstack.guest.v1`, and the proto says so explicitly. `AttestGpu`'s proto comment contrasted itself against `GpuInfo`, a method that exists on no surface: the real contrast is `Attest`'s `boottime_gpu_evidence`, which is the thing that returns a boot record. Documentation gaps: the spec pointed at `rpc_service_v1/keys.rs` for the private-key vectors and the v0-forgery regression test, both of which live in `ra-tls/src/api_v1.rs` -- a reader checking the normative bytes landed where they are not. The Errors table omitted the two validations `AttestGpu` performs. And the `Health` fail-open for an app that never opted into gating -- `healthy: true` meaning "nobody asked me to know" -- was pinned only by a test, in none of the three documents that describe the method. Tests: `IssueCert` and `AttestGpu` had no handler-level coverage, so nothing would have noticed the v1 handlers ceasing to route through the shared validity check or the attestor. Both are now asserted through the handler, and the on-demand GPU format tag is pinned like its boot-time counterpart.
…nature The comment said a malleated signature "fails to parse rather than verifying". Measured against k256 0.13.4: `Signature::from_slice` accepts the high-S bytes -- it only rejects an `r` or `s` outside `1..n`, and `n - s` is still in range -- and the rejection happens inside verification. So a caller gets HTTP 200 with `valid: false`, not the 400 a parse failure produces. The security answer is unchanged and the behaviour is right; only the explanation was wrong, and it was wrong about the status code, which is the part a client branches on.
… 400 The doc comment on the frozen `Verify` makes an observable claim: k256 rejects high-S inside the verification rather than in `from_slice`, so a malleated signature comes back as HTTP 200 with `valid: false`, not as the 400 a parse failure would produce. Nothing pinned it. The test builds the malleation by negating `s`, then checks it is the real thing before using it -- `normalize_s` is `Some` only for a high-S signature, and it must normalise back to exactly the signature that just verified -- so the assertions cannot pass against a no-op. Both halves of the claim are asserted: `.expect()` for the status code a 0.5.x client branches on, `!valid` for the security answer. A k256 upgrade that moved the check into parsing would keep the second and silently break the first. That is the drift the comment warns about.
Three edits from review of the preceding two commits. `GetAttestationForAppKey`'s comment said the report data was "the same commitment the frozen `Sign` path's key uses". `Sign` uses a key, not a commitment. Say what is actually shared: the derivation -- same path, purpose and base algorithm -- so the attested public key is the one that signs. The Errors table now lists a row that no client can clear by changing its request. Since a 400 in this API means "the method ran and failed" rather than "your request was malformed", say so where the table says everything above is a 400, and say that the message in the body is what separates the two. `app-health-checks.md` restated the fail-open mechanism the proto comment and the spec already carry. Keep the reading an operator needs -- what `true` means for an app that opted out -- and drop the third copy of how it is computed. Also reflow the paragraph the `ra-tls` pointer fix left short of the fill.
… GPU
`AttestGpu` reported "GPU attestation is not available in this image" as an
uncoded error, which `dispatch_prpc` turns into the generic 400. That tells a
client its request was malformed. It was not: the request is well-formed, and
no other request would succeed either, because the image ships no nvattest and
will not grow one at runtime. A client branching on the status retries with
different arguments forever instead of falling back.
`ra_rpc::ErrorExt::with_code` already carries a chosen status through the
transport, and `code_of` walks the whole error chain, so the code survives the
handler's `.context("GPU attestation failed")`. 501 rather than 503 because the
capability is absent for the lifetime of the CVM, not temporarily unavailable.
A malformed nonce keeps the default 400 -- that one really is the caller's
fault, and the two failures must not be indistinguishable.
Safe to change now: `AttestGpu` is v1-only and never shipped in a release, so
no deployed client is reading the old 400. `rpc_service_v1.rs` already asserts
it is absent from both frozen surfaces.
The availability probe moves from `nvattest::available()` to a `GpuAttestor`
field holding the binary's path. Without that, a test for the unavailable
answer would pass only on a host with no nvattest installed and would spawn a
real collection against the host's GPUs anywhere else. The test fixture pins it
to a path that cannot exist, so every guest-agent test sees the same answer.
Documented in the proto comment, the spec's Errors and Status codes sections,
and the curl API reference, which listed only 400 and 500.
`return Err(anyhow!("...").with_code(501))` is four moving parts to say one
thing, and it reads as an error construction rather than as a control-flow
exit -- the two existing sites both wrote it differently, one importing
`ErrorExt` and one calling it as a free function.
`ra_rpc::bail!(501, "...")` expands to exactly that expression. The format
arguments are forwarded to `anyhow::anyhow!` verbatim, so nothing new has to be
learned, and the macro resolves `anyhow` through a hidden re-export so a caller
does not need it in scope.
The code is the first argument and is mandatory. An error with no status to
choose should keep using `anyhow::bail!` and be reported as CODE_BAD_REQUEST,
which is the correct default; requiring the code here means the two spellings
say which kind of failure they are. `gpu_attest.rs` now shows both in one
function -- a bad nonce bails plainly, a missing nvattest bails with 501.
Converted both existing sites. The 413 path in `read_data` is already covered
end to end by `status_codes::a_payload_over_the_limit_is_rejected`, so the
macro reaching the transport is pinned without a new test; the unit test added
here covers the part that test cannot see, that the code survives the
`.context(..)` a handler adds on the way out.
kvinwang
enabled auto-merge
August 25, 2026 02:02
kvinwang
added a commit
that referenced
this pull request
Aug 25, 2026
`AttestGpu` on an image that ships no nvattest answers 501, not 400 -- the request is well-formed and no retry of it will ever succeed, so a client that reads 4xx retries forever while one that reads 501 falls back. The agent started saying so in #1118; these tests still asserted the old 400, and the JS one failed against a `next` that had moved. Widened rather than relaxed. The JS assertion now names 501 and the agent's own words instead of matching any 4xx, and the Rust and Python tests -- which only checked that *something* failed -- pin the status too. That is this branch's own claim under test: every SDK reports a non-2xx response with both the server's error text and the HTTP status, and until now nothing proved it for the one status that tells a caller to stop trying.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Follow-up to #1116. A post-merge audit of the whole v1 surface — agent, proto, spec, and the four SDKs — turned up five more comments of the species that PR was already cleaning up, three documentation gaps, and two of the six v1 methods with no handler-level test. This is the agent/proto/spec half; the SDK half is a separate PR.
Comments naming things that do not exist
GetAttestationForAppKey's doc still credited "the v1 method" with sharing its report data, and the test pinning its vectors still saidWorkerV1wraps those bytes in an attestation. v1 ships neither: there is deliberately no v1AttestAppKey, asagent_rpc_v1.protoand the spec both say at length two paragraphs earlier.Four more places wrote
WorkerV1as a service name. The service isWorkerin packagedstack.guest.v1— the proto is explicit that the names carry no version suffix, and the spec says to write "the v1Worker". The Rust import aliasesWorkerV1Client/WorkerV1Serverare untouched: those disambiguate two same-named generated types in one scope, which is the reason they exist.AttestGpu's proto comment contrasted itself againstGpuInfo, a method that exists on no surface — it was removed from the unversioned surface as a never-released addition, andrpc_service_v1.rsasserts it is absent from both. The real contrast isAttest'sboottime_gpu_evidence, which is the thing that returns a record written at boot.Documentation gaps
The spec pointed at
rpc_service_v1/keys.rsfor the private-key vectors and fora_v0_claim_cannot_be_crafted_into_a_v1_claim. Both live inra-tls/src/api_v1.rs;keys.rspins only the public-key column and says so itself. A reader following either pointer to check the normative bytes landed where they are not.The Errors table omitted the two validations
AttestGpuactually performs (nonce length, and nvattest being absent from the image), while listing the other four.Health's fail-open for an app that never opted into gating —healthy: truemeaning "nobody asked me to know", not "checked and fine" — was pinned only by a test, and appeared in none of the three documents that describe the method. It is now in the proto comment, the spec, andapp-health-checks.md, each saying what a non-gateway caller should read thattrueas.Tests
IssueCertandAttestGpuwere the two v1 methods with no handler-level coverage, so nothing would have noticed the v1 handlers ceasing to route through the shared validity check or through the attestor. Both are now asserted through the handler rather than against the helper in isolation, and the on-demand GPU format tag is pinned like its boot-time counterpart already was — a consumer selects its verifier on(vendor, format), so both tags are wire contract.The
AttestGputest formats the error with{:#}deliberately: the handler adds a context line, andra_rpcencodes errors asformat!("{error:#}"), so asserting on the flatto_string()would test something no client ever sees.Verification
cargo test -p dstack-guest-agent --all-features: 118 passed (3 new).dstack-gateway: 289 passed.frozen_surface: 2 passed — the frozen descriptor digest is unmoved, as a comment-only proto edit requires.cargo fmt --all --checkandcargo clippy -D warnings(per CLAUDE.md) clean.